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REMARKS 

The specification and claims have been amended to clarify the invention. In particular, the 
title of the specification has been amended to recite "NUCLEIC ACIDS ENCODING A TNF 
RECEPTOR 2 RELATED PROTEIN". Claim 2 has also been amended to delete reference to 
"fragments" of SEQ ID NO:2. Claim 3 has been amended to recite a cDNA encoding a protein 
of claim 1, and claim 5 has been amended to depend from amended claim 3. No new matter is 
added by any of these amendments, and entry of the amendments is therefore requested. 

The examiner stated that claims 1-7 are currently under examination. 
Objection to the specification 

The examiner has objected to the title of the invention as not descriptive. A new title is 
required that is clearly indicative of the invention to which the claims are directed, and suggested 
the following title: Nucleic acids encoding TNF receptor 2 related protein. The title has been so 
amended. 

35 U.S.C.S 1 12, First Paragraph, Rejection of Claims L 2 and 4-7 

The examiner has rejected claims 1, 2 and 4-7 under 35 U.S.C. § 1 12, first paragraph, as 
containing subject matter which was not described in the specification in such a way as to 
reasonably convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. 

The examiner stated that the claims as written include nucleic acids comprising fragments 
and homologues or nucleic acids encoding polypeptides comprising fragments and homologues, 
and encompass polypeptides that vary substantially in length and also in amino acid composition. 
The instant disclosure of a single polypeptide, SEQ ID N0:1, with instantly disclosed specific 
activity does adequately support the scope of the claimed genus, which encompasses a substantial 
variety of sub genera. The examiner cited various case law in support of the requirement for 
adequate written description under 35 U.S.C. § 112, first paragraph, in particular: Regents of the 
University of California v Eli Lilly & Co, 119F3d 1559, 1569, 43 USPQ2d 1398, 1406 (Fed Cir. 
1997); and Hers v. Revel 984 F.2d 1164, 1171, 25 USPQ2d 1601, 1606 (Fed. Cir. 1993). 

The examiner further stated that a description of a genus of cDNAs may be achieved by 
means of a recitation of a representative number of cDNAs, defined by nucleotide sequence, 
falling within the scope of the genus, or a recitation of structural features common to the genus, 
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which features constitute a substantial portion of the genus. The Examiner stated that the instant 
specification discloses, however, a single polypeptide sequence of SEQ ED N0:1 and the highly 
related human and mouse LTp-R proteins. However, the Examiner stated, protein function 
cannot be reliably predicted from protein sequence homology (citing Vukicevic et al., 1996; 
Tischer et al., U.S. Patent No. 5,194,596; and Kopchick et al., U.S. Patent No. 5,350,836). Given 
the unpredictability of homology comparisons and the fact that the specification fails to provide 
objective evidence that the additional sequences are indeed species of the claimed genus, it can 
not be established that a representative number of species have been disclosed to support the 
genus. The instantly claimed genus is not limited to having a specific activity and the prior art 
does not provide compensatory structural or correlative teachings to enable one of skill in the art 
to identify the polynucleotides encompassed. 
Applicants Response 

Claim 2 has been amended to delete fragment language, and claim 1 recites only one 
specific fragment encoded by a polynucleotide of the invention. Thus the claimed fragments of 
the polynucleotides of the invention are adequately described. With regard to the claimed 
"variants" or "homologues" of the polynucleotide of SEQ ID NO:2 and polypeptide of SEQ ID 
NO:l, applicants disagree that they are not described in sufficient terms that one of skill in the art 
would recognize applicants possession of them in accordance with the written description 
requirements of 35 U.S.C. § 112, first paragraph. 

The requirements necessary to fulfill the written description requirement of 35 U.S.C. 1 12, 

first paragraph, are well established by case law. 

... the applicant must also convey with reasonable clarity to those skilled 
in the art that, as of the fihng date sought, he or she was in possession of the 
invention. The invention is, for purposes of the "written description" inquiry, 
whatever is now claimed, Vas-Cath, Inc. v. Mahurkar, 19 USPQ2d 1111, 1117 
(Fed. Cir. 1991) 

Attention is also drawn to the Patent and Trademark Office's own "Guidelines for 

Examination of Patent Applications Under the 35 U.S.C. Sec. 112, para. 1", published January 5, 

2001, which provide that : 

An applicant may also show that an invention is complete by disclosure of 
sufficiently detailed, relevant identifying characteristics which provide evidence 
that applicant was in possession of the claimed invention, i.e., complete or partial 
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structure, other physical and/or chemical properties, functional 
characteristics when coupled with a known or disclosed correlation between 
function and structure, or some combination of such characteristics. What is 
conventional or well known to one of ordinary skill in the art need not be 
disclosed in detail. If a skilled artisan would have understood the inventor to be in 
possession of the claimed invention at the time of filing, even if every nuance of 
the claims is not explicitly described in the specification, then the adequate 
description requirement is met (emphasis added). 

Thus, the written description standard is fulfilled by both what is specifically disclosed 
and what is conventional or well known to one skilled in the art. 

SEQ ID N0:1 and SEQ ID NO:2 are specifically disclosed in the application (see, for 
example, page 2, hnes30-33). Variants of SEQ ID NO:l are described, for example, at page 3, 
line 26, and of SEQ ID N0:2 at page 9, lines 35-37. Specific structural characteristics of 
polynucleotide and polypeptide variants of the invention are described at page 8, lines 14-19. 
Incyte clones in which the nucleic acids encoding the human TNFR2PV were first identified and 
libraries from which those clones were isolated are described, for example, at page 8, line 24 
through page 9, line 4 of the Specification. Chemical and structural features of TNFR2PV are 
described, for example, on page 9, lines 6-24. Given SEQ ID NO:l, one of ordinary skill in the 
art would recognize naturally-occurring variants of SEQ ID NO:l having at least 95% sequence 
identity to SEQ ID NO:l. Accordingly, the Specification provides an adequate written 
description of the recited polypeptide sequences. 

A. The SpeciGcation provides an adequate written description of the claimed "variants" of 
SEQ ID NO:l. 

The Office Action has further asserted that the claims are not supported by an adequate 

written description because 

Given the unpredictability of homology comparisons, and the fact that the 
specification fails to provide objective evidence that the additional sequences are 
indeed species of the claimed genus, it cannot be established that a representative 
number of species have been disclosed to support the genus claim. 

(page 6 of the Office Action of 6/17/2003) 
Such a position is believed to present a misappUcation of the law. 
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1. The present claims speciHcally define the claimed genus through the 

recitation of chemical structure 

Court cases in which "DNA claims" have been at issue (which are hence relevant to 

claims to proteins encoded by the DNA) commonly emphasize that the recitation of structural 

features or chemical or physical properties are important factors to consider in a written 

description analysis of such claims. For example, in Fiers v. Revel, 25 USPQ2d 1601, 1606 

(Fed. Cir. 1993), the court stated that: 

If a conception of a DNA requires a precise definition, such as by structure, 
formula, chemical name or physical properties, as we have held, then a description 
also requires that degree of specificity. 

In a number of instances in which claims to DNA have been found invalid, the courts 

have noted that the claims attempted to define the claimed DNA in terms of functional 

characteristics without any reference to structural features. As set forth by the court in University 

of California v. Eli Lilly and Co,, 43 USPQ2d 1398, 1406 (Fed. Cir. 1997): 

In claims to genetic material, however, a generic statement such as "vertebrate 
insulin cDNA" or "manmialian insulin cDNA," without more, is not an adequate 
written description of the genus because it does not distinguish the claimed genus 
from others, except by function. 

Thus, the mere recitation of functional characteristics of a DNA, without the definition of 
structural features, has been a conmion basis by which courts have found invalid claims to DNA. 
For example, in Lilly, 43 USPQ2d at 1407, the court found invalid for violation of the written 
description requirement the following claim of U.S. Patent No. 4,652,525: 

1. A recombinant plasmid replicable in procaryotic host containing within its 
nucleotide sequence a subsequence having the structure of the reverse transcript of 
an mRNA of a vertebrate, which mRNA encodes insulin. 

In Fiers, 25 USPQ2d at 1603, the parties were in an interference involving the following 

count: 

A DNA which consists essentially of a DNA which codes for a human fibroblast 
interferon-beta polypeptide. 

Party Revel in the Fiers case argued that its foreign priority application contained an 
adequate written description of the DNA of the count because that application mentioned a 
potential method for isolating the DNA. The Revel priority application, however, did not have a 
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description of any particular DNA structure corresponding to the DNA of the count. The court 

therefore found that the Revel priority application lacked an adequate written description of the 
subject matter of the count. 

Thus, in Lilly and Fiers, nucleic acids were defined on the basis of functional 
characteristics and were found not to comply with the written description requirement of 35 
U.S.C. §112; Le,, "an mRNA of a vertebrate, which mRNA encodes insuUn" in Lilly, and "DNA 
which codes for a human fibroblast interferon-beta polypeptide" in Fiers, In contrast to the 
situation in Lilly and Fiers, the claims at issue in the present application define polynucleotides 
or polypeptides in terms of chemical structure, rather than on functional characteristics. For 
example, the "variant language" of independent claim 1 recites chemical structure to define the 
claimed genus: 

1. An isolated cDNA... comprising a nucleic acid sequence encoding a protein 
selected from:...b) a naturally-occurring variant of the amino acid sequence of 
SEQ ID NO:l having at least 95% identity to SEQ ID NO:l. 

From the above it should be apparent that the claims of the subject appUcation are 
fundamentally different from those found invalid in Lilly and Fiers. The subject matter of the 
present claims is defined in terms of the chemical structure of SEQ ID N0:1. In the present case, 
there is no reliance merely on a description of functional characteristics of the polypeptides 
recited by the claims. In fact, there is no recitation of functional characteristics. Moreover, if 
such functional recitations were included, it would add to the structural characterization of the 
recited polypeptides. The polypeptides defined in the claims of the present application recite 
structural features, and cases such as Lilly and Fiers stress that the recitation of structure is an 
important factor to consider in a written description analysis of claims of this type. By failing to 
base its written description inquiry "on whatever is now claimed," the Office Action failed to 
provide an appropriate analysis of the present claims and how they differ from those found not to 
satisfy the written description requirement in Lilly and Fiers 

2. The present claims do not define a genus which is highly variant 

Furthermore, the claims at issue do not describe a genus which could be 
characterized as "highly variant" or which "vary substantially in length or composition." 
Available evidence illustrates that the claimed genus is of narrow scope. 
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In support of this assertion, the Examiner's attention is directed to the enclosed 
reference by Brenner et al. ("Assessing sequence comparison methods with rehable structurally 
identified distant evolutionary relationships," Proc. Natl. Acad. Sci. USA (1998) 95:6073-6078). 
See also specification, page 27, lines 6-7. Through exhaustive analysis of a data set of proteins 
with known structural and functional relationships and with <90% overall sequence identity, 
Brenner et al. have determined that 30% identity is a reliable threshold for establishing 
evolutionary homology between two sequences aligned over at least 150 residues. (Brenner et 
al., pages 6073 and 6076.) Furthermore, local identity is particularly important in this case for 
assessing the significance of the alignments, as Brenner et al. further report that ^40% identity 
over at least 70 residues is reliable in signifying homology between proteins. (Brenner et al., 
page 6076.) ^ 

The present application is directed, inter alia, to TNF R2 related proteins, in 
particular, variants of the human TNF R2 related protein LTRp-R (g339762; Figures 2A-2C); 
proteins related to the amino acid sequence of SEQ ID N0:1. In accordance with Brenner et al, 
naturally occurring molecules may exist which could be characterized as TNF R2 related proteins 
and which have as little as 40% identity over at least 70 residues to SEQ ID NO: 1. The "variant 
language" of the present claims recites, for example, polynucleotides encoding "a naturally- 
occurring amino acid sequence having at least 95% sequence identity to the sequence of SEQ ID 
N0:1" (note that SEQ ED N0:1 has 399 amino acid residues). This variation is far less than that 
of all potential TNF R2 related proteins related to SEQ ID N0:1, i.e., those TNF R2 related 
proteins having as little as 40% identity over at least 70 residues to SEQ ID N0:1. 

Furthermore, based on the teachings of Brenner, applicants dispute the 
Examiner's allegation that protein function cannot be reliably predicted from protein sequence 
homology. While the Examiner has cited literature identifying some of the difficulties that may 
be involved in predicting protein function, none suggest that functional homology cannot be 
inferred by a reasonable probability in this case. Importantly, none contradict Brenner's basic 
rule that sequence homology in excess of 40% over 70 or more amino acid residues yields a high 
probability of functional homology as well. Nor do they contradict the characterization of 
TNFR2PV and related variants in terms of the cysteine-rich domains common to TNFR proteins 
(see page 1, lines 31-34; and page 9, lines 11-17). At most, these articles individually and 
together stand for the proposition that it is difficult to make predictions about function with 
certainty. The standard applicable in this case is not, however, proof to certainty, but rather proof 
to reasonable probability. 
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3. The state of the art at the time of the present invention is further advanced 
than at the time of the Lilly and Fiers applications 

In the Lilly case, claims of U.S. Patent No. 4,652,525 were found invalid for failing to 
comply with the written description requirement of 35 U.S.C. §112. The '525 patent claimed the 
benefit of priority of two applications, Apphcation Serial No. 801,343 filed May 27, 1977, and 
Application Serial No. 805,023 filed June 9, 1977. In the Fiers case, party Revel claimed the 
benefit of priority of an Israeli application filed on November 21, 1979. Thus, the written 
description inquiry in those case was based on the state of the art at essentially at the "dark ages" 
of recombinant DNA technology. 

The present application was filed in August 2001. Much has happened in the 
development of recombinant DNA technology in the 20 or more years from the time of filing of 
the applications involved in Lilly and Fiers and the present application. For example, the 
technique of polymerase chain reaction (PGR) was invented. Highly efficient cloning and DNA 
sequencing technology has been developed. Large databases of protein and nucleotide sequences 
have been compiled. Much of the raw material of the human and other genomes has been 
sequenced. With these remarkable advances one of skill in the art would recognize that, given 
the sequence information of SEQ ID NO:l and SEQ ID N0:2, and the additional extensive detail 
provided by the subject application, the present inventors were in possession of the claimed 
polynucleotide variants at the time of filing of this application. 

4. Sununary 

The Office Action failed to base its written description inquiry "on whatever is now 
claimed." Consequently, the Action did not provide an appropriate analysis of the present claims 
and how they differ from those found not to satisfy the written description requirement in cases 
such as Lilly and Fiers. In particular, the claims of the subject application are fundamentally 
different from those found invalid in Lilly and Fiers, The subject matter of the present claims is 
defined in terms of the chemical structure of SEQ ID NO: 1 or SEQ ID N0:2. The courts have 
stressed that structural features are important factors to consider in a written description analysis 
of claims to nucleic acids and proteins. In addition, the genus of polynucleotides or polypeptides 
defined by the present claims is adequately described, as evidenced by Brenner et al and 
consideration of the claims of the '740 patent involved in Lilly, Furthermore, there have been 
remarkable advances in the state of the art since the Lilly and Fiers cases, and these advances 
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were given no consideration whatsoever in the position set forth by the Office Action. 

Withdrawal of the rejection of claims under 35 U.S.C. 1 12, first paragraph is therefore requested. 
35 U.S.C. § 112, Second Paragraph, Rejection of Claims 4 and 7 

The Examiner has rejected claims 4 and 7 under 35 U.S.C. § 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. In particular, the Examiner stated that claim 4 is indefinite 
because it encompasses a composition comprising a cDNA and a labeling moiety, and it is not 
clear if the labeling moiety is covalently attached to the cDNA or is separate. Also, the Examiner 
stated, claim 7 is indefinite because it encompasses a method of using the cDNA of claim 1 to 
produce a protein using the complement of the cDNA encoding SEQ ID NO: 1 which would not 
encode the protein of SEQ ID NO: 1. Thus, it is not clear what protein would be produced. 
A pplicants Response 

Applicants disagree that the specification does adequately convey to the skilled artisan the 

nature of the composition comprising the cDNA and a labeling moiety. In addition to the fact 

that the nature of a "labeling moiety" in conjunction with a cDNA probe or protein are well 

known in the art, the specification describes a "labeling moiety" at page 6, lines 33-36 as follows: 

"Labeling moiety" refers to any visible or radioactive label than can be attached to or 
incorporated into a cDNA or protein. Visible labels include but are not limited to 
anthocyanins, green fluorescent protein (GFP), 6 glucuronidase, luciferase, Cy3 and Cy5, 
and the like. Radioactive markers include radioactive forms of hydrogen, iodine, 
phosphorous, sulfur, and the like. 

Thus, both the nature of the interaction between the cDNA probe and the labeling moiety 
are described as "attached to or incorporated into a cDNA", and specific probes, the manner of 
attachment of which are well known in the art, are described. Their manner of attachment would 
therefore be clear to the skilled artisan. 

With regard to polynucleotides "complementary" to the cDNAs encoding SEQ ID N0:1 
and which proteins would be of indeterminate composition, claim 3 has been amended to recite 
only polynucleotides encoding "a protein of claim 1", and claim 7 now depends from claim 3. 
The protein(s) produced by the method of claim 7 are therefore clear and definite. 

Withdrawal of the rejection of claim 4 and 7 under 35 U.S.C. § 112, second paragraph is 
therefore requested. 

35 U.S.C. § 102(b), Rejection of Claims L 2 and 4-7 
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The Examiner has rejected claims 1, 2 and 4-7 under 35 U.S.C. § 102(b) as being 

anticipated by Baens et al. (Genomics, Vol 16, pp 214-218, 1993). The Examiner stated that 
Baens et al. disclose a cDNA that is 91.3% identical to SEQ ID N0:2 and is therefore a variant 
having at least 85% identity to SEQ ID NO:2. Furthermore, since the cDNA of Baens et al. 
comprises a nucleic acid sequence of SEQ ID NOs:7-9 (ranging from nucleotides 629-1378 of 
SEQ ID N0:2), Baens also meets the limitation of claim 2(b). Claim 2(b) does not have a 
limitation that the nucleic acid sequences are contiguous. The cDNA of Baens et al. also encodes 
a protein identical to SEQ ID NO:l except for a missing 36 amino acid segment. Examiner 
contends, that since claim 1 recites "An isolated cDNA... comprising a nucleic encoding a 
protein", and there is no limitation that the nucleic acid encodes contiguous amino acids, the 
cDNA of Baens et al. meets the limitations of the claims. The Examiner further stated that 
Baens et al. also teach the limitations of claims 4-6. The Examiner concluded that this rejection 
could be overcome by adding limitations in which the sequences must be contiguous. 
Applicants Response 

Applicants disagree that the claims, as presently amended, do not convey that the recited 
sequences are "contiguous" or "continuous". Applicants disagree that the common meaning of 
the term "sequence" does not imply a "contiguous" or "continuous" succession of nucleic acid or 
amino acid residues in, for example, the phrase "an amino acid sequence of SEQ ID NO:l" as 
recited in claim 1. The American Heritage Dictionarv , Second College Addition (Houghton 
Mifflin Co., 1985), for example, defines "sequence" as: 

1. A follov^ing of one thing after the other; succession. 2. An order of 
succession; arrangement. 3. A related or continuous series.... 

Thus a "sequence", as it is commonly understood in the art, is a continuous succession of 
recited elements. As to the recitation of "An isolated cDNA... comprising a nucleic encoding a 
protein", § 2111.03 of the MPEP clearly defines the term "comprising" as meaning that the 
named elements (e.g., SEQ ID NO:l or SEQ ID NO:2) are "essential", but that other elements 
(additional sequence) may be added. Since, as the Examiner has noted, the cDNA of Baens et al. 
encodes a protein identical to SEQ ID NO:l except for a missing 36 amino acid segment, clearly 
the cDNA of Baens et al. does not contain the essential "named elements" of the claim and 
therefore does not "comprise" a cDNA encoding the recited protein. 
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For all of the above reasons, applicants submit that the cDNA of Baens et al. does not 

anticipate a cDNA "comprising a nucleic acid of SEQ ID N0:2", or "comprising a nucleic acid 
sequence encoding.... SEQ ID NOrl". Withdrawal of the rejection of claims 1, 2 and 4-7 under 
35 U.S.C. § 102(b) as being anticipated by Baens et al. is therefore requested. 
35 U.S.C. 103(ay Rejection of Claim 7 

The Examiner has rejected claim 7 as unpatentable over Baens et al. in view of Sibson et 
al (WO 94/01548). The teachings of Baens et al are summarized. Baens does not teach 
recombinant production of a protein. However, the Examiner stated Sibson et al. disclose 
placing a desired cDNA sequence into an expression vector and a host cell to express a protein, 
and it would therefore have been prima facie obvious to one of skill in the art at the time the 
invention was made to use the cDNA of Baens et al. in an expression vector and a host cell to 
express the isolated polypeptide. 
Applicants Response 

Applicants arguments rebutting the anticipation of the claimed polynucleotides and 
polypeptides of the invention as recited in claims 1-3, by Baens et al. are sunmiarized above. 
Therefore, neither Baens et al. nor Sibson et al. teach or suggest SEQ E) NO:l, SEQ ED NO:2, 
the recited variants of SEQ ID NO: 1 or SEQ ID N0:2, or the antigenic fragment of SEQ ID 
N0:1 from about amino acid residue P216 to about amino acid residue P235 of SEQ ID NO:l. 
Applicants submit that for there to be a proper prima facie case of obviousness against a claim, 
the reference, or combination of references, must teach or suggest all of the claim limitations. 
MPEP § 2143.03. There is therefore no proper prima facie case of obviousness against claim 7 
(which now depends from claim 3), and withdrawal of the rejection is requested. 
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CONCLUSION 

In light of the above amendments and remarks. Applicants submit that the present 
application is fully in condition for allowance, and request that the Examiner withdraw the 
outstanding objections/rejections. Early notice to that effect is earnestly solicited. Applicants 
further request that, upon allowance of claims 1 and 4, claims 8-13 be rejoined and examined as 
methods of use of the polynucleotides of claims 1 and 4 that depend from and are of the same 
scope as these claims, in accordance with in Re Ochiai and the MPEP § 821.04. 

. If the Examiner contemplates other action, or if a telephone conference would expedite 
allowance of the claims, Applicants invite the Examiner to contact the undersigned at the number 
listed below. 

Applicants believe that no fee is due with this conmiunication. However, if the USPTO 
determines that a fee is due, the Conmiissioner is hereby authorized to charge Deposit Account 
No. 09-0108. 
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ABSTRACT Pairwisc sequence comparison methods have 
been assessed using proteins whose relationships are known 
reliably from their structures and functions, as described in 
the SCOP database [Murzin, A. G., Brenner, S. £., Hubbard, T. 
& Cbothia C. (1995) 7. Afo/. Biol, 2A7, 536-540]. The evalua- 
tion tested the programs BLAST {Altscbul, S. Gish, 
Miller, W., Myers, E. W. & Lipman, D. J. (1990). J. Mot, Biol, 
215, 403-410), WU-BLACT2 [Altschul, S, F. & Gish, W. (1996) 
Methods Enzymol. 266, 460-480], FACTA [Pearson, W. R. & 
Upman, D. J. (1988) Proc. Natl, Acad, Sci. USA 85, 2444-2448], 
and SSEARCH [Smith, T. F. & WaUrman, M. S. (1981) /. Mol, 
Bid, 147, 195-197] and their scoring schemes. The error rate 
of all algorithms is greatly reduced by using statistical scores 
to evaluate matches rather than percentage identity or raw 
scores. The £-valuc statistical scores of SSEARCH and Fasta are 
reliable: the number of false positives found in our tests agrees 
well with the scores reported. However, the P-values reported 
by BLAST and wu-blaST2 exaggerate signiTicance by orders of 
magnitude, ssearch, fasta ktup =: 1, and wu-blasT2 perform 
best, and they are capable of detecting almost all relationships 
between proteins whose sequence identities are >30%. For 
more distantly related proteins, they do much less well; only 
one-half of the relationships between proteins with 20-30% 
identity are found. Because many homologs have low sequence 
similarity, most distant relationships cannot be detected by 
any pairwise comparison method; however, those which art 
identified may be used with confidence. 



Sequence database searching plays a role in virtually every 
branch of molecular biology and is crucial for inierpreling the 
sequences issuing forth from genome projects. Given the 
method's central role« it is surprising that overall and relative 
capabilities of different procedures are largely unknown. It is 
difficult to verify algorithms on sample data because this 
requires large data sets of proteins whose evolutionary rela- 
tionships are known unambiguously and independently of the 
methods being evaluated. However, nearly all known ho- 
mologs have been identified by sequence analysis (the method 
to be tested). Also, it is generally very difficult to know, in the 
absence of structural data, whether two proteins that lack clear 
sequence similarity are unrelated. This has meant that al- 
though previous evaluations have helped improve sequence 
comparison, they have suffered from insufficient, imperfectly 
characterized, or artificial lest data. Assessment also has been 
problematic because high quality database sequence searching 
attempts to have both sensitivity (deieaion of homologs) and 
specificity (rejecUon of unrelated proteins); however, these 
complementary goals are linked such that increasing one 
causes the other to be reduced. 
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Sequence comparison methodologies have evolved rapidly, 
so no previously published tests has evaluated modern versions 
of programs commonly used. For example, parameters in 
BLAST (1) have changed, and wu-blast2 (2)— which produces 
gapped alignments — has become available. The latest version 
of FASTA (3) previously tested was 1.6, but the current release 
(version 3.0) pix>vides fundamentally different results in the 
form of statistical scoring. 

The previous reports also have left gaps in our knowledge. 
For example, there has been no published assessment of 
thresholds for scoring schemes more sophisticated than per- 
centage identity. Thus, the widely discussed statistical scoring 
measures have never actually been evaluated on large data- 
bases of real proteins. Moreover, the different scoring schemes 
commonly in use have not been compared. 

Beyond these issues, there is a more fundamental question: 
in an absolute sense, how well does pairwise sequence com- 
parison work? That is. what fraction of homologous proteins 
can be detected using modem database searching methods? 

In this work, we attempt to answer these questions and to 
overcome both of the fundamental difficulties that have hin- 
dered assessment of sequence comparison methodologies. 
First, we use the set of distant evolutionary relationships in the 
SCOP: Structural Classification of Proteins database (4), which 
is derived from structural and functional characteristics (5). 
The SCOP database provides a uniquely reliable set of ho- 
mologs, which are known independently of sequence compar- 
ison. Second, we use an assessment method that jointly mea- 
sures both sensitivity and specificity. This method allows 
straightforward comparison of different sequence searching 
procedures. Further, it can be used to aid interpretation of real 
database searches and thus provide optimal and reliable 
results. 

Previous Assessments of Sequence Comparison. Several 
previous studies have examined the relative performance of 
different sequence comparison methods. The most encom- 
passing analyses have been by Pearson (6, 7), who compared 
the three most commonly used programs. Of these, the Smith- 
Waterman algorithm (8) implemented in ssearch (3) is the 
oldest and slowest but the most rigorous. Modem heuristics - 
have provided BLAST (1) the speed and convenience to make 
it the most popular program. Intermediate between these two 
is fasta (3), which may be run in two modes offering either 
greater speed (ktup = 2) or greater effectiveness (ktup - 1). 
Pearson also considered different parameters for each of these 
programs. 

To test the methods, Pearson selected two representative 
proteins from each of 67 protein superfamilies defmed by the 
PIR database (9). Each was used as a query to search the 
database, and the matched proteins were marked as being 
homologous or unrelated according to their membership of PIR 
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superfamilies. Pearson found that modem matrices and "In- 
scaiing'* of raw scores improve results considerably. He also 
reported that the rigorous Smith-Waterman algorithm worked 
slightly better than fast a, which was in turn more effective 
than BLAST. 

Very large scale analyses of matrices have been performed 
(10), and Henikoff and Henikoff (11) also evaluated the 
effectiveness of BLAST and fasta Their lest with blast 
considered the ability to detect homologs above a predeter- 
mined score but had no penally for methods which also 
reported large numbers of spuriotis matches. The Henikoff s 
searched the swiss-PROT database (12) and used PROSrrE (13) 
to define homologous families. Their results showed that the 
BLOSUM62 matrix (14) performed markedly bener than the 
extrapolated PAM-series matrices (15), which previously had 
been popular. 

A crucial aspect of any assessment is the data that are used 
lo test the ability of the program to find homologs. But in 
Pearson's and the Henikoffs* evaluations of sequence com- 
parison, the correct results were effectively unknown. This is 
because the super families in PIR and PROSfTE are principally 
created by using the same sequence comparison methods 
which are being evaluated, interdependency of data and 
methods creates a "chicken and egg" problem, and means for 
example, that new methods would be penalized for correctly 
identifying homologs missed by older programs. For instance, 
immunoglobulin variable and constant domains are clearly 
homologous, but PIR places them in different super families. 
The problem is widespread: each superfamily in PIR 48.00 with 
a structural homolog is itself homologous to an average of 1.6 
other PIR superfamilies (16). 

To surmount these sorts of difficulties, Sander and Schnei- 
der (17) used protein structures to evaluate sequence com- 
parison. Rather than comparing different sequence compari- 
son algorithms, their work focused on determining a length- 
dependent threshold of percentage identity, above which all 
proteins would be of similar structure. A result of this analysis 
was the HSSP equation; it slates that proteins with 25% identity 
over 80 residues will have similar structures, whereas shorter 
alignments require higher identity. (Other studies also have 
used structures (18-20), but these focused on a small number 
of model proteins and were principally oriented toward eval- 
uating alignment accuracy rather than homology detection.) 

A general solution to the problem of scoring comes from 
statistical measures (i.e., E-values and P- values) based on the 
extreme value distribution (21). Extreme value scoring was 
implemented analytically in the blast program using the 
Karlin and Altschul statistics (22, 23) and empirical ap- 
proaches have been recently added to fasta and ssearch. In 
addition to being heralded as a reliable means of recognizing 
significantly similar proteins (24, 25), the mathematical trac- 
tability of statistical scores "is a crucial feature of the blast 
algorithm" (1). The validity of this scoring procedure has been 
tested analytically and empirically (see ref. 2 and references in 
ref. 24). However, all large empirical tests used random 
sequences that may lack the subtle structure found within 
biological sequences (26, 27) and obviously do not contain any 
real homologs. Thus, although many researchers have sug- 
gested that statistical scores be used to rank matches (24, 25, 
28), there have been no large rigorous experiments on biolog- 
ical data to determine the degree to which such rankings are 
superior. 

A Database fer Testing Homology Detection. Since the 
discovery that the structures of hemoglobin and myoglobin are 
very similar though their sequences are not (29), it has been 
apparent that comparing structures is a more powerful (if less 
convenient) way to recognize distant evolutionary relation- 
ships than comparing sequences. If two proteins show a high 
degree of similarity in their structural details and function, it 



is very probable that they have an evolutionary relationship 
though their sequence similarity may be low. 

The recent growth of protein structure information com- 
bined with the comprehensive evolutionary classification in 
the SCOP database (4, 5) have aUowed us to overcome previous 
limitations. With these data, we can evaluate the performance 
of sequence comparison methods on real protein sequences 
whose relationships arc known confidently. The scop database 
uses structural information to recognize distant homologs, the 
large majority of which can be determined unambiguously. 
These superfamilies, such as the globins or the immunoglobu- 
lins, would be recognized as related by the vast majority of the 
biological community despite the lack of high sequence sim- 
ilarity. 

From SCOP, we extracted the sequences of domains of 
proteins in the Protein Data Bank (PDB) (30) and created two 
databases. One (PDB90D-B) has domains, which were all <90% 
identical to any other, whereas (PDB40D-B) had those <40% 
identical. The databases were created by first sorting all 
protein domains in SCOP by their quality and making a list. The 
highest quality domain was selected for inclusion in the 
database and removed from the list. Also removed from the list 
(and discarded) were all other domains above the threshold 
level of identity to the seleaed domain. This process was 
repeated until the list was empty. The PDB40D-B database 
contains 1,323 domains, which have 9,044 ordered pairs of 
distant relationships, or -*0^% of the total 1,749,006 ordered 
pairs. In PDB90D-B, the 2,079 domains have 53,988 relation- 
ships, representing 1.2% of all pairs. Low complexity regions 
of sequence can achieve spurious high scores, so these were 
masked in both databases by processing with the SEG program 
(27) using recommended parameters: 12 1.8 2.0. The databases 
used in this paper are available from http://sss.stanford.edu/ 
sss/, and databases derived from the current version of SCOP 
may be found at http://scop.mrc-lmb.cam.ac-uk/scop/. 

Analyses from both databases were generally consistent, but 
PDB40D-B focuses on distantly related proteins and reduces the 
heavy overrepresentation in the PDB of a small number of 
families (31. 32), whereas PDB90D-B (with more sequences) 
improves evaluations of statistics. Except where noted other- 
wise, the distant homolog results here are from PDB40D-B. 
Although the precise numbers reported here are specific to the 
structural domain databases used, we expect the trends to be 
general. 

Assessment Data and Procedure. Our assessment of se- 
quence comparison may be divided into four different major 
categories of tests. First, using just a single sequence compar- 
ison algorithm at a time, we evaluated the effectiveness of 
different scoring schemes. Second, we assessed the reliability 
of scoring procedures, including an evaluation of the validity 
of statistical scoring. Third, we compared sequence compari- 
son algorithms (using the optimal scoring scheme) to deter- 
mine their relative performance. Fourth, we examined the 
distribution of homologs and considered the power of pairwisc 
sequence comparison to recognize them. All of the analyses - 
used the databases of structurally identified homologs and a 
new assessment criterion. . 

The analyses tested BLAST (1), version 1.4.9MP, and wu- 
BLflLST2 (2), version 2.0al3MP. Also assessed was the Fasta 
package, version 3.0t76 (3), which provided FASTA and the 
SSEARCH implementation of Smith-Waterman (8). For 
SSEARCH and fasta, we used BLOSUM45 with gap penalties 
-12/-1 (7, 16). The default parameters and matrix (BLO- 
SUM62) were used for blast and wu-bu>lST2. 

The "Coverage Vs. Error" Plot To test a particular protocol 
(comprising a program and scoring scheme), each sequence 
from the database was used as a query to search the database. 
This yielded ordered pairs of query and target sequences with 
associated scores, which were sorted, on the basis of their 
scores, from best to worst. The ideal method would have 
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Fig. 1 . Coverage vs. error plots of different scoring schemes for ssearCH Smith-Waterman. (A ) Analysis of PDB40D-B database. (B) Analysis 
of PDB90D-B database. All of the proteins in the database were compared with each other using the ssearch program. The results of this single 
set of comparisons were considered using five different scoring schemes and assessed. The graphs show the coverage and errors per query (EPQ) 
for statistical scores, raw scores, and three measures using percentage identity. In the coverage vs. error plot, the x axis indicates the fraction of 
all homologs in the database (known from structure) which have been deteaed. Precisely, it is the number of detected pairs of proteins with the 
same fold divided by the total number of pairs from a common super family. PDB40i>-6 contains a total of 9.044 homologs, so a score of 10% indicates 
identification of 904 relationships. The y axis reports the number of EPQ. Because there are 1323 queries made in the PDB40D-B aU-vs.-all 
comparison, 13 errors corresponds to 0.01, or 1% EPQ. They axis is presented on a log scale to show results over the widely varying degrees of 
accuracy which may be desired. The scores that correspond to the levels of EPQ and coverage are shown in Fig. 4 and Table 1. The graph 
demonstrates the trade-off between sensitivity and selectiviiy. As more homologs are found (moving to the right), more errors are made (moving 
up). The ideal method would be in the lower right comer of the graph, which corresponds to identifying many evolutionary relationships without 
selecting unrelated proteins. Three measures of percentage identity are plotted. Percentage identity within alignment is the degree of identity within 
the aligned region of the proteins, without consideration of the alignment length. Percentage identity within both is the number of identical residues 
in the aligned region as a percentage of the average length of the query and target proteins. The HSSP equation (17) is H = 290.l5/~**-*" where 
/ is length for 10 < / < 80; H > 100 for / < 10; H 24.7 for / > 80. The percentage identity Hssp-adjusted score is the percent identity within 
the alignment minus H. Smith- Water man raw scores and E-values were taken directly from the sequence comparison program. 



perfect separation, with all of the homologs at the top of the 
list and unrelated proteins below. In practice, perfect separa- 
tion is impossible to achieve so instead one ts interested in 
drawing a threshold above which there are the largest number 
of related pairs of sequences consistent with an acceptable 
error rale. 

Our procedure involved measuring the coverage and error 
for every threshold. Coverage was defined as the fraction of 
structurally determined homologs that have scores above the 
selected threshold; this reflects the sensitivity of a method. 
Errors per query (EPQ)» an indicator of selectivity, is the 
number of nonhomologous pairs above the threshold divided 
by the number of queries. Graphs of these data, called 
coverage vs. error plots, were devised to understand how 



protocols compare at different levels of accuracy. These 
graphs share effectively all of the beneficial features of Re- 
ciever Operating Characteristic (ROC) plots (33. 34) but 
belter represent the high degrees of accuracy required in 
sequence comparison and the huge background of nonho- 
mologs. 

This assessment procedure is directly relevant to practical 
sequence database searching, for it provides precisely the 
information necessary to perform a reliable sequence database 
search. The EPQ measure places a premium on score consis- 
tency; that is, it requires scores to be comparable for different 
queries. Consistency is an aspect which has been largely 

Percent Identity of Unrelated Proteins (PDB90D-B) 
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Fic. 2. Unrelated proteins with high percentage identity. Hemo- 
globin ^-chain (PDB code Ihds chain b, ref. 38. Left) and cellulase E2 
(PDB code 1 tml, rcf. 39, Right) have 39% identity over 64 residues, a 
level which is often believed to be indicative of homology. Despite this 
high degree of identity, (heir structures strongly suggest that these 
proteins are not related. Appropriately, neither the raw alignment 
score of 85 nor the E-value of 1.3 is significant. Proteins rendered by 
RASMOL (40). 
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Fig. 3. Length and percentage identity of alignments of unrelated 
proteins in PDB90l>-B: Each pair of nonhomologous proteins found with 
ssearch is plotted as a point whose position indicates the length and 
the percentage identity within the alignment. Because alignment 
length and percentage identity arc quantized, many pairs of proteins 
may have exactly the same alignment length and percentage identity. 
The line shows the hssp threshold (though it is intended to be applied 
with a different matrix and parameters). 
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Fig. 4, Reliability of statistical scores in PDB90D-B: Each line shows 
the relationship between reported statistical score and actual error 
rate for a different program. E-valucs arc reported for ssearch and 
FASTA, whereas P-values arc shown for BL^sr and wu-blash. If the 
scoring were perfect, then the number of errors per query and the 
E-valucs would be the same, as indicated by the upper bold line. 
(P-values should be the same as EPO for small numbers, and diverges 
at higher values, as indicated by the lower bold line.) E-valucs from 
SSEARCH and fasta arc shown to have good agreement with EPO but 
underestimate the significance slightly, blast and wu-BlAsn are 
overconfident, with the degree of exaggeration dependent upon the 
score. The results for pdB40D-b were similar to those for pdB90I>-b 
despite the difference in number of homologs detected. This graph 
could be used to roughly calibrate the reliability of a given statistical 
score. 

ignored in previous tests but is essential for the straightforward 
or automatic interpretation of sequence comparison results. 
Further, it provides a clear indication of the confidence that 
should be ascribed to each match. Indeed, the EPQ measure 
should approximate the expectation value reported by data- 
base searching programs, if the programs' estimates are accu- 
rate. 

The PcrformaDce of Scoring Schemes. All of the programs 
tested could provide three fundamental types of scores. The 
first score is the percentage identity, which may be computed 
in several ways based on either the length of the alignment or 
the lengths of the sequences. The second is a "raw" or 
"Smith-Waterman" score, which is the measure optimized by 
the Smith-Waterman algorithm and is computed by summing 
the substitution matrix scores for each position in the align- 
ment and subtracting gap penalties. In Blj\ST. a measure 

Sequence Comperison Algorithms (PDB40D-B) 
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related to this score is scaled into bits. Third is a statistical 
score based on the extreme value distribution. These results 
arc summarized in Fig. 1. 

Sequence Identity. Though it has been long established that 
percentage identity is a poor measure (35). there is a common 
rule-of-thumb stating that 30% identity signifies homology. 
Moreover, publications have indicated that 25% identity can 
be used as a threshold (17, 36). We find that these thresholds, 
originally derived years ago, are not supported by present 
results- As databases have grown, so have the possibilities for 
chance alignments with high identity; thus, the reported cutoffs 
lead to frequent errors. Fig. 2 shows one of the many pairs of 
proteins with very different structures^that nonetheless have 
high levels of identity over considerable aligned regions. 
Despite the high identity, the raw and the statistical scores for 
such incorrect matches are typically not significant. The prin- 
cipal reasons percentage identity does so poorly seem to be 
that it ignores information about gaps and about the conser- 
vative or radical nature of residue substitutions. 

From the PDB90l>B analysis in Fig. 3. we learn that 30% 
identity is a reliable threshold for this database only for 
sequence alignments of at least 150 residues. Because one 
unrelated pair of proteins has 43.5% identity over 62 residues, 
it is probably necessary for alignments to be at least 70 residues 
in length before 40% is a reasonable threshold, for a database 
of this particular size and composition. 

At a given reliability, scores based on percentage identity 
delect just a fraction of the distant homologs found 
statistical scoring. If one measures the percentage identity in 
the aligned regions without consideration of alignment length, 
then a negligible number of distant homologs are detected. 
Use of the hssp equation improves the value of percentage 
identity, but even this measure can find only 4% of all known 
homologs at 1% EPQ. In short, percentage identity discards 
most of the information measured in a sequence comparison. 

Raw Scores. Smith-Waterman raw scores perform bener 
than percentage identity (Fig. 1), but In-scaling (7) provided no 
notable benefit in our analysis. It is necessary to be very precise 
when using either raw or bit scores because a 20% change in 
cutoff score could yield a tenfold difference in EPQ. However, 
it is difficult to choose appropriate thresholds because the 
reliability of a bit score depends on the lengths of the proteins 
matched and the size of the database. Raw score thresholds 
also are affected by matrbc and gap parameters. 

Statistical Scores. Statistical scores were introduced partly 
to overcome the problems that arise from raw scores. This 
scoring scheme provides the best discrimination between 
homologous proteins and those which are unrelated. Most 
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Fig. 5. Coverage vs. error plots of different sequence comparison methods: Five different sequence comparison methods arc evaluated, each 
using statistical scores (E- or P-values). {A ) PD640D-B database. In this analysis, the best method is the slow ssearch, which finds 18% of relationships 
at 1% EPO- FASTA kiup - 1 and wu-BLast2 are almost as good. (B) pdb90I>B database. The quick wu-blast7 program provides the best coverage 
at 1% EPQ on (his database, although at higher levels of error it becomes slightly worse than fasta ktup = 1 and ssearch. 
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likely» its power can be attributed to its incorporation of more 
information than any other measure; it takes account of the 
fuU substitution and gap data (like raw scores) but also has 
details about the sequence lengths and composition and is 
scaled appropriately. , . ^ . 

We find that statistical scores are not only powerful, but also 
easy to interpret. sseaRCH and Fasta show close agreement 
between statistical scores and actual number of errors per 
query (Fig- 4). The expectation value score gives a good, 
slightly conservative estimate of the chances of the two se- 
quences being found at random in a given query. Thus, an 
E-value of 0.01 indicates that roughly one pair of nonhomologs 
of this simUarity should be found in every 100 different queries. 
Neither raw scores nor percentage identity can be interpreted 
in this way. and these results validate the suitability of the 
extreme vzdue distribution for describing the scores from a 
database search. 

The P-values from BLAST also should be directly interpret- 
able but were found to overstate significance by more than two 
orders of magnitude for 1% EPQ for this database. Nonethe- 
less, these results strongly suggest that the analytic theory is 
fundamentally appropriate. wu-BLASn scores were more re- 
liable than those from blast, but also exaggerate expected 
confidence by more than an order of magnitude at 1% EPQ. 

Overall Detection of Homologs and Comparison of Algo* 
ntbms. The results in Fig. 5A and Table 1 show that pairwise 
sequence comparison is capable of identifying only a small 
fraction of the homologous pairs of sequences in PDB40D-B. 
Even SSEARCH with E-values, the best protocol tested, could 
find only 18% of all relationships at a 1% EPQ. blast, which 
identifies 15%. was the worst performer, whereas fasta 
ktup = 1 is nearly as effective as ssEARCH. fasta ktup = 2 and 
WU-BLAST2 are intermediate in their ability to detect ho- 
mologs. Comparison of different algorithms indicates that 
those capable of identifying more homologs are generally 
slower. SSEARCH is 25 times slower than BLAST and 6.5 limes 
slower than Fasta ktup = 1. wu-biast2 is slightly faster than 
FASTA ktup = 2, but the latter has more interpretable scores. 

In PDB90D-B, where there are many close relationships, the 
best method can identify only 38% of structurally known 
homologs (Fig. 5B). The method which finds that many 
relationships is wv-hLflSU. Consequently, we infer that the 
differences between faSTa kup = 1, SSEARCH. and wu-blast2 
programs are unlikely to be significant when compared with 
variation in database composition and scoring reliability. 

Fig. 6 helps to explain why most distant homologs cannot be 
found by sequence comparison: a great many such relation- 
ships have no more sequence identity than would be expected 
by chance. ssEARCH with E-values can recognize >90% of the 
homologous pairs with 30-40% identity. In this region, there 
are 30 pairs of homologous proteins thai do not have signif- 
icant E-values, but 26 of these involve sequences with <50 
residues. Of sequences having 25-30% identity. 75% are 
identified by SSEARCH E-values. However, although the num- 
ber of homologs grows at lower levels of identity, the detection 
falls off sharply: only 40% of homologs with 20-25% identity 
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Fig. 6. Distribution and detection of homologs in pdB40I>-b. Bars 
show the distribution of homologous pairs PDB40O-B according to ihcir 
identity (using the measure of identity in both). Filled regions indicate 
the number of these pairs found by the best database searching method 
(SSEARCH with E-values) at \% EPQ. The pdb40D-B database contains 
proteins with <40% identity, and as shown on this graph, most 
structurally identified homologs in the database have diverged ex- 
tremely far in sequence and have <20% identity. Note that the 
alignments may be inaccurate, especially at low levels of identity. Filled 
regions show that ssearch can identify most relationships that have 
25% or more identity, but its detection wanes sharply below 25%. 
Consequently, the great sequence divergence of most structurally 
identified evolutionary relationships effectively defeats the ability of 
pariwise sequence comparison to detect them. 

are detected and only 10% of those with 15-20% can be found. 
These results show that statistical scores can find related 
proteins whose identity is remarkably low; however, the power 
of the method is restricted by the great divergence of many 
protein sequences. 

After completion of this work, a new version of pairwise 
BLAST was released: blastgp (37). It supports gapped align- 
ments, like WU-BLAST2, and dispenses with sum statistics. Our 
initial tests on blastgp using default parameters show that its 
E-values are reliable and that its overall detection of homologs 
was substantially better than that of ungapped BLAST, but not 
quite equal to that of WU-BLAST2. 

CONCLUSION 

The general consensus amongst experts (see refs. 7. 24. 25. 27 
and references therein) suggests that the most effective se- 
quence searches are made by {/) using a large currerit database 
in which the protein sequences have been complexity masked 
and (h) using statistical scores to interpret the results. Our 
experiments fully support this view. 

Our results also suggest two further points. First, the E-val- 
ues reported by FASTA and ssearch give fairly accurate 
estimates of the significance of each match, but the P-values 
provided by blast and wu-blast2 underestimate the true 



Table I. Summary of sequence comparison methods with pdB40I>-B 


Method 


Relative Time* 


1% EPO Cutoff 


Coverage al 1% EPO 


SSEARCH % identity: within alignment 
SSEARCH % identity: within both 
SSEARCH % identity: HSSP-scaled 
ssearch Smith- Waterman raw scores 
SSEARCH E-valucs 
FASTA ktup = 1 E-values 
FASTTA klup = 2 E-valucs 
WU-BLAST2 p-values 
BLAST P-valucs 


253 
253 
25-5 
25.5 
253 
3.9 
1.4 
1.1 
1.0 


>70% 
34% 

35% (HSSP + 9.8) 
142 
0.03 
0.03 
0.03 
0.003 
0.00016 


<0.1 
3.0 
4.0 
10.5 
18.4 
17.9 
16.7 
17.5 
14.8 


•Times are from large database searches with genome proteins. 
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extent of errors. Second, ssearch, wu-blast2, and fasta 
ktup = 1 perform best, though blast and fasta ktup = 2 
detect most of the relaiionships found by the best procedures 
and are appropriate for rapid initial searches. 

The homologous proteins that are found by sequence com- 
parison can be distinguished with high reliability from the huge 
number of unrelated pairs. However, even the best database 
searching procedures tested fail to find the large majority of 
distant evolutionary relationships at an acceptable error rate. 
Thus, if the procedures assessed here faO to find a reliable 
match, it does not imply that the sequence is unique; rather, it 
indicates that any relatives it might have are distant ones.** 

••Additional and updated information about this work, including 
supplementary figures, may be found at htt p://ss5^tanford.edu/sss/ 
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